Layer Test
The book has now been published and the content of this chapter has likely changed substanstially.Please see page 337 of xUnit Test Patterns for the latest information.
Also known as: Single Layer Test, Testing by Layers, Layered Test
How can we verify logic independently when it is part of a Layered Architecture?
We write separate tests for each layer of the Layered Architecture.
Sketch Layer Tests embedded from Layer Tests.gif
It is difficult to get good test coverage when testing an entire application top-to-bottom; we are bound to end up doing Indirect Testing (see Obscure Test on page X) on some parts of the application. Many applications use a Layered Architecture[DDD,PEAA,WWW] to separate the major technical concerns. Most applications have some kind of presentation (user interface) layer, a business logic layer or domain layer and a persistence layer. Some layered architectures have even more layers.
An application with a layered architecture can be tested more effectively by testing each layer in isolation.
How It Works
We design the system under test (SUT) using a layered architecture that separates the presentation logic from the business logic from any persistence mechanism or interfaces to other systems. (not all presentation logic is user interface; it can also be a messaging interface used by another application) We put all business logic into a Service Layer[PEAA] that exposes the application functionality to the presentation layer as an API. We treat each layer of the architecture as a separate SUT. We write component tests for each layer independent of the other layers of the architecture. That is, for layer n of the architecture, the tests will take the place of layer n+1; we may optionally replace layer n-1 with a Test Double (page X).
When To Use It
We can use a Layer Test whenever we have a Layered Architecture and we want to get good test coverage of the logic in each layer. It can be much simpler to test each layer independently than it is to try to test all the layers at once. This is especially true when we want to do defensive coding for return values of calls across the layer boundary. In software that is working correctly, these errors "should never happen"; in real life, they do. We can make sure our code handles them by injecting these "never happen" scenarios as indirect inputs of our layer.
Layer Tests are very useful when we want to divide up the project team into sub teams based on the technology they specialize in. Each layer of an architecture tends to require different knowledge and often use different technologies; therefore, the layer boundaries are natural team boundaries. Layer Tests can be a good way to nail down and document the semantics of the layer interfaces.
Even when we choose to use a Layer Test strategy, it is good to have a few "top-to-bottom" tests just to verify that the various layers are actually integrated correctly. These tests only need to cover one or two basic scenarios; we don't need to test every business test condition this way since they have already been tested in the Layer Tests for at least one of the layers.
Most of the variations on this pattern come down to which layer is being tested independently of the other layers.
Variation: Presentation Layer Test
One could write a whole book just on patterns of presentation layer testing. The specific patterns would depend on the nature of the presentation layer technology being used (e.g. graphical user interface, traditional web interface, "smart" web interface, etc.) Regardless of the technology, the key is to try to test the presentation logic separately from the business logic so that we don't have to worry about changes in the underlying logic affecting our presentation layer tests. (They are hard enough to automate well as it is!)
Another consideration is to design the presentation layer so that the presentation layer logic can be tested independently of the presentation framework. Humble Dialog (see Humble Object on page X) is the key design for testability pattern to apply here. In effect, we are defining sub-layers within the presentation layer; the layer containing the Humble Dialogs is the "presentation graphic layer" while the layer we have made testable is the "presentation behavior layer". This allows us to verify that buttons are activated, menu items are grayed out, etc. without actual instantiating any of the real graphical objects.
Variation: Service Layer Test
The Service Layer is where most of our unit tests and component tests are traditionally concentrated. Testing the business logic using customer tests is a bit more challenging because testing the Service Layer via the presentation layer often involves the use of Indirect Testing and Sensitive Equality (see Fragile Test on page X) either of which can lead to Fragile Tests and High Test Maintenance Cost (page X). Testing the Service Layer directly helps avoid this.
To keep from having Slow Tests (page X) we usually run these tests with the persistence layer replaced by a Fake Database (see Fake Object on page X). In fact, most of the impetus behind a Layered Architecture is to isolate this code from the other, harder-to-test layers. Alistair Cockburn has an interesting spin on this in the form of a Hexagonal Architecture[WWW].
The Service Layer may come in handy for other uses. It can be used to run the application in "headless" mode (without a presentation layer attached) such as when using macros to automate frequently done tasks in Microsoft Excel.
Variation: Persistence Layer Test
The persistence layer also needs to be tested. Round trip tests will often suffice if the application is the only one that uses the data store. One kind of programming error that these tests won't catch is when we accidentally put information into the wrong columns. As long as the data type of the interchanged columns is compatible and we make the same error when reading the data, our round trip tests will pass! This kind of bug won't affect the operation of our application but it might make support harder and it will cause problems in interactions with other applications.
When other applications also use the data store it is highly advisable to have at least a few layer-crossing tests to verify that information is being put into the correct columns of tables. We can use Back Door Manipulation (page X) to either set up the database contents or to verify the post-test database contents.
Variation: Subcutaneous Test
A Subcutaneous Test is a degenerate form of Layer Test that bypasses the presentation layer of the system to interact directly with the Service Layer. In most cases, the Service Layer is not isolated from the layer(s) below; therefore, we are testing everything except the presentation. Use of a Subcutaneous Test does not require as strict a separation of concerns as does a Service Layer Test which makes it easier to use when retrofitting tests onto an application that wasn't designed for testability. We should use a Subcutaneous Test whenever we are writing customer tests for an application and we want to ensure our tests are robust. A Subcutaneous Test is much less likely to be broken by changes to the application (than a test that exercises the logic via the presentation layer) because it does not interact with it via the presentation layer and therefore a whole category of changes won't affect it.
Variation: Component Test
A Component Test is the most general form of Layer Test in that we can think of the layers being made up of individual components that act as "micro-layers". Component Tests are a good way to specify or document the behavior of individual components when we are doing component-based development and some of the components have to be modified or built from scratch.
Implementation Notes
We can write our Layer Tests as either round trip tests or layer-crossing tests. Each has advantages and in practice, we typically mix both styles of tests. The round trip tests are easier to write (assuming we already have a suitable Fake Object available to use for layer n-1) but we need to use layer-crossing tests when verifying the error handling logic in layer n.
Round Trip Tests
A good starting point for Layer Tests is the round trip test. They should be sufficient for most Simple Success Tests (see Test Method on page X). These can be written such that they do not care whether we have fully isolated the layer from the layers below. We can either leave the real components in place so that they are exercised indirectly, or we can replace them with Fake Objects. The latter is particularly useful when we have Slow Tests caused by a database or asynchronous mechanisms in the layer below.
Controlling Indirect Inputs
We can replace a lower layer of the system with a Test Stub (page X) that returns "canned" results based on what the client layer passes in a request. (E.g. Customer 0001 is a valid customer, 0002 is a dormant customer, 0003 has 3 accounts, etc.) This allows us to test the client logic with well understood indirect inputs from the layer below. This is particularly useful when automating Expected Exception Tests (see Test Method) or when exercising behavior that depends on data that arrives from an upstream system (typically directly into a shared database or via a "data pump"). The alternative is to use Back Door Manipulation to set up the indirect inputs.
Verifying Indirect Outputs
When we want to verify the indirect outputs of the layer under tests, we can use a Mock Object (page X) or Test Spy (page X) to replace the components in the layer below the SUT. This allows us to compare the actuals calls made to the depended-on component (DOC) with what we expected. The alternative is to use Back Door Manipulation to verify the indirect outputs of the SUT after they have occurred.
Motivating Example
When trying to test all the layers of the application at the same time, we have to verify the correctness of the business logic through the presentation layer. The following test is a very simple example of testing some trivial business logic through a trivial user interface.
private final int LEGAL_CONN_MINS_SAME = 30;
public void testAnalyze_sameAirline_LessThanConnectionLimit()
throws Exception {
// setup
FlightConnection illegalConn = createSameAirlineConn( LEGAL_CONN_MINS_SAME - 1);
// exercise
FlightConnectionAnalyzerImpl sut = new FlightConnectionAnalyzerImpl();
String actualHtml = sut.getFlightConnectionAsHtmlFragment( illegalConn.getInboundFlightNumber(),
illegalConn.getOutboundFlightNumber());
// verification
StringBuffer expected = new StringBuffer();
expected.append("<span class=”boldRedText”>");
expected.append("Connection time between flight ");
expected.append(illegalConn.getInboundFlightNumber());
expected.append(" and flight ");
expected.append(illegalConn.getOutboundFlightNumber());
expected.append(" is ");
expected.append(illegalConn.getActualConnectionTime());
expected.append(" minutes.</span>");
assertEquals("html", expected.toString(), actualHtml);
}
Example MultiLayerTest embedded from java/com/clrstream/ex10/student/test/FlightConnectionAnalyzerTest.java
This test contains knowledge about the business layer functionality (what makes a connection illegal) and presentation layer functionality (how an illegal connection is presented.) It also depends on the database because the FlightConnections are retrieved from another component. If any of these areas change, this test will need to be revisited.
Refactoring Notes
We can split this test into two separate tests, one to test the business logic (what constitutes an illegal connection) and one to test the presentation layer (given an illegal connection, how should it be displayed to the user?) We would typically do this by duplicating the entire Testcase Class (page X) and stripping out the presentation layer logic verification from the business layer Test Methods and stubbing out the business layer object(s) in the presentation layer Test Methods.
Along the way, we will probably find that we can reduce the number of tests in at least one of the Testcase Classes because there are few test conditions for that layer. In this example, we started out with 4 tests (the combinations of same/different airline and time periods) that each test both business and presentation layer; we ended up with 4 tests in the business layer (the original combinations but tested directly), and only two presentation layer tests, legal and illegal. (I'm glossing over the various error-handling tests to simplify this discussion but Layer Test also makes it easier to exercise the error-handling logic.) Therefore, only the latter two tests need to be concerned about the details of the string formatting and when a test fails, we know which layer the bug is in.
We can go even farther by using a Replace Dependency with Test Double (page X) refactoring to turn this Subcutaneous Test into a true Service Layer Test.
Example: Presentation Layer Test
Here's the test refactored to verify the behavior of the presentation layer when an illegal connection is requested. It stubs out the FlightConnAnalyser and configures it with the illegal connection to return to the HtmlFacade when it is called. This give us complete control over the indirect input of the SUT.
public void testGetFlightConnAsHtml_illegalConnection()
throws Exception {
// setup
FlightConnection illegalConn = createIllegalConnection();
Mock analyzerStub = mock(IFlightConnAnalyzer.class);
analyzerStub.expects(once()).method("analyze").will(returnValue(illegalConn));
HTMLFacade htmlFacade = new HTMLFacade( (IFlightConnAnalyzer)analyzerStub.proxy());
// exercise
String actualHtmlString = htmlFacade.getFlightConnectionAsHtmlFragment(
illegalConn.getInboundFlightNumber(),
illegalConn.getOutboundFlightNumber());
// verify
StringBuffer expected = new StringBuffer();
expected.append("<span class=”boldRedText”>");
expected.append("Connection time between flight ");
expected.append(illegalConn.getInboundFlightNumber());
expected.append(" and flight ");
expected.append(illegalConn.getOutboundFlightNumber());
expected.append(" is ");
expected.append(illegalConn.getActualConnectionTime());
expected.append(" minutes.</span>");
assertEquals("returned HTML", expected.toString(),
actualHtmlString);
}
Example PresentationLayerTest embedded from java/com/clrstream/ex10/solution/test/HTMLFacadeTest.java
We must compare the string representations of the HTML to determine whether the correct response has been generated. Fortunately, we only need two such test to verify the basic behavior of this component.
Example: Subcutaneous Test
Here's the original test converted into a Subcutaneous Test that bypasses the presentation layer to verify that the connection information is calculated correctly. Note the lack of any string manipulation in this test.
private final int LEGAL_CONN_MINS_SAME = 30;
public void testAnalyze_sameAirline_LessThanConnectionLimit()
throws Exception {
// setup
FlightConnection expectedConnection =
createSameAirlineConn( LEGAL_CONN_MINS_SAME -1);
// exercise
IFlightConnAnalyzer theConnectionAnalyzer = new FlightConnAnalyzer();
FlightConnection actualConnection = theConnectionAnalyzer.getConn( expectedConnection.getInboundFlightNumber(),
expectedConnection.getOutboundFlightNumber());
// verification
assertNotNull("actual connection", actualConnection);
assertFalse("IsLegal", actualConnection.isLegal());
}
Example SubcutaneousTest embedded from java/com/clrstream/ex10/solution/test/FlightConnectionAnalyzerTest.java
While we have bypassed the presentation layer, we are not doing anything to isolate the Service Layer from the layers below so this could result in Slow Tests or Erratic Tests (page X).
Example: Business Layer Test
Here's the same test converted into a Service Layer Test that is fully isolated from the layers below it. We have used JMock to replace these components with Mock Objects that verify the right flights are being looked up and which inject the corresponding flight constructed into the SUT.
public void testAnalyze_sameAirline_EqualsConnectionLimit()
throws Exception {
// setup
Mock flightMgntStub = mock(FlightManagementFacade.class);
Flight firstFlight = createFlight();
Flight secondFlight = createConnectingFlight( firstFlight, LEGAL_CONN_MINS_SAME);
flightMgntStub.expects(once()).method("getFlight")
.with(eq(firstFlight.getFlightNumber()))
.will(returnValue(firstFlight));
flightMgntStub.expects(once()).method("getFlight")
.with(eq(secondFlight.getFlightNumber()))
.will(returnValue(secondFlight));
// exercise
FlightConnAnalyzer theConnectionAnalyzer = new FlightConnAnalyzer();
theConnectionAnalyzer.facade = (FlightManagementFacade)flightMgntStub.proxy();
FlightConnection actualConnection = theConnectionAnalyzer.getConn( firstFlight.getFlightNumber(),
secondFlight.getFlightNumber());
// verification
assertNotNull("actual connection", actualConnection);
assertTrue("IsLegal", actualConnection.isLegal());
}
Example BusinessLayerTest embedded from java/com/clrstream/ex10/solution/test/FlightConnectionAnalyzerTest.java
This test runs very quickly because the Service Layer has been fully isolated from any underlying layers. It is likely to be much more robust because it is testing much less code.
Copyright © 2003-2008 Gerard Meszaros all rights reserved